Field of the Invention
The present invention relates to image analysis and, more particularly, to methods for chromogen separation-based image analysis related to quantitative video-microscopy techniques in cellular biology and pathology applications.
Description of Related Art
The assessment and analysis of tissues is the domain of pathology. During the recent past, methodological and technological developments have turned digital image analysis into one of the most efficient tools to assist pathologists in interpreting images with increased accuracy. Though such image analysis techniques contribute substantially to provide cytologists with accurate, reproducible and objective cellular analysis, histological interpretation techniques still tend to depend on the subjective analysis of specimens. Such histological interpretation techniques may also be subject to varying intra- as well as inter-observer agreement, which further tend to provide less accurate, less reproducible, and less objective results. For such reasons, image analysis of tissues was initially restricted to technologies developed for the analysis of cytological specimens.
With the evolution and availability of high performance computers, local and wide area communication, cost-effective database solutions, improved storage technology, and cost-effective high-resolution digital cameras and/or scanners, the situation has now changed. More sophisticated algorithms, formerly ineffective due to lack of CPU power, could not before be applied to tissue sections in a routine environment. However, such algorithms can now be used to assess and quantify tissue-specific features related to marker quantification and sub cellular localization. At the same time, more comprehensive support for a reproducible and more standardized visual assessment of tissue sections has become available based on the initial step in image analysis, namely the creation and management of digital images. This is especially true in the fields of quality control, quality assurance and standardization. Digital images of difficult cases can be exchanged with reference pathologists via telepathology to get a second opinion. Such images can also be effectively used for proficiency testing. Digital images are also the basis of powerful image reference databases, which can be accessed via network, and play an increasingly important role in the documentation of cases and evaluation results, particularly in comprehensive electronic or printed reports.
Once a tissue slide is prepared, a pathologist visually examines the tissue specimen under a microscope. If image analysis should be applied with respect to the slide, the microscope must be at least equipped with a camera or other image capturing device, which is connected to a computer system via an interface. The camera samples the optical microscopic image of the tissue sample via the microscope. As a result, a digital image is collected in the memory of the computer and can be displayed on the monitor thereof. However, the acquisition of these digital images must be performed such that the important details of the optical images are still correctly represented by the stored data.
Generally, the next step for a quantitative assessment of the digitized images is segmentation, which sometimes includes an additional intermediate step of preprocessing. During segmentation, the cells are separated from each other and from the image background. In some instances, algorithmic advances have made it possible to segment cells down to the sub-cellular component level (i.e., nucleus, cytoplasm, and membranes). Although it may appear an easy task, segmentation is often a difficult and error-prone step in image analysis. For slides where the cells are nicely separated and stained in a way that good contrasts occur in the digitized image, segmentation can be done very reliably in many cases. As soon as one of the above conditions is not fulfilled, however, highly sophisticated and time consuming segmentation algorithms, using additional a priori knowledge about the cells and their relationship to each other, or about marker and counter stain sub-cellular localization, have to be applied. This is the case, for example, in instances of tissue sections of infiltrating tumors, where most of the cells are no longer nicely separated on the slide, but tend to be touching and overlapping each other.
Using a marker-based algorithm, it is possible to circumscribe the region of interest automatically, and let the pathologist decide, using his own subjective expertise, if the region presented is adequate or needs to be manually refined. Once the meaningful areas of an image are determined, the feature extraction takes place. For each cell (and its sub-cellular components), a set of densitometric, morphometric, texture, and contextual features can be measured, with a goal of characterizing the individual cells and their interactions as comprehensively as possible.
The last step is the presentation of the raw data and compilation thereof into meaningful results and/or scores. The resulting output of an image analysis system should desirably match the form of visual and/or semi-quantitative grading systems already in use by the pathologist so as to promote consistency, to be easily applicable, or to be capable of being interpreted in routine use.
The platform for the evaluation of tissue samples via image analysis is shifting more and more from the general-purpose image analyzer to specialized and dedicated “pathology workstations” configured for routine work. Such workstations combine tools needed to provide the pathologist with the necessary information to derive the best results possible. Central to such a workstation is the microscope, possibly equipped with robotic parts including a motorized stage, an automatic focus device, an objective changer, and a light intensity adjustment device. Different input devices, such as cameras capable of fast automatic focusing and acquisition of high resolution images, are linked to the workstation. The workstation can be part of a Local Area Network (LAN). The workstation can also support different communication protocols, so that available communication channels can be used to connect the workstation with other places in the world (Wide Area Network or WAN).
When integrated within a LAN and/or WAN, the workstation can be granted access to existing reference databases and Hospital Information Systems (HIS) such that any new cases to be examined can be compared with the pictures and accompanying information of reference cases which have been accumulated over time. In addition, images acquired from the slides under review can be complemented with the patient and case history.
The pathology workstation is preferably suited for a comprehensive tissue evaluation. Starting with information and digital pictures of the initial tissue sample, images of the slides prepared from the tissue can be taken. The patient and case information, the images themselves, and any quantitative information about the cell components of the tissue sample can all be stored in the same database.
All of the information accumulated by the workstation for one case, such as images, measurement results, patient data, preparation data, can be selected to be part of a report which can either be printed or signed out electronically via the network. The report provides a comprehensive picture of the case under evaluation and facilitates quality assurance and standardization.
During preprocessing/segmentation of the captured images, many different techniques/algorithms can be implemented for image analysis, particularly for quantitative video-microscopy in the field of cellular biology and pathology applications, by using multi-spectral imaging adapted to color cameras (i.e., RGB 3CCD cameras).
Effective analysis of microscopic images is essential in cellular biology and pathology, particularly for detection and quantification in genetic material (genes, messenger RNA) or the expression of this genetic information in the form of proteins, for example, gene amplification, gene deletion, gene mutation, number of messenger RNA molecules or protein expression analyses. Gene amplification is the presence of too many copies of the same gene in one cell, wherein a cell usually contains two copies, otherwise known as alleles, of the same gene. Gene deletion indicates that less than two copies of a gene can be found in a cell. Gene mutation indicates the presence of incomplete or non-functional genes. Messenger RNAs (mRNA) are molecules of genetic information, synthesized from gene reading, that serve as templates for protein synthesis. Protein expression is the production of a given protein by a cell. If the gene coding for this protein is up regulated or too many copies of the gene or mRNA are present, the protein may be over-expressed. If the gene is down regulated or deleted, the protein expression level may be low or absent.
Normal cellular behaviors are precisely controlled by molecular mechanisms involving a large number of proteins, mRNAs and genes. Gene amplification, gene deletion, and gene mutation are known to have a prominent role in abnormal cellular behaviors through abnormal protein expression. The range of cellular behaviors of concern includes behaviors as diverse as, for example, proliferation or differentiation regulation. Therefore, effective detection and quantification in gene amplification, deletion and mutation, mRNAs levels or protein expression analyses, is necessary in order to facilitate useful research, diagnostic and prognostic tools.
There are numerous laboratory techniques dedicated to detection and quantification in gene amplification, deletion and mutation, mRNA levels or protein expression analyses. For example, such techniques include Western, Northern and Southern blots, polymerase chain reaction (“PCR”), enzyme-linked immunoseparation assay (“ELISA”), and comparative genomic hybridization (“CGH”) techniques. However, microscopy is routinely utilized because it is an informative technique, allowing rapid investigations at the cellular and sub-cellular levels, which may be implemented at a relatively low cost.
When microscopy is the chosen laboratory technique, the biological samples usually first undergo specific detection and revelation preparations. Once the samples are prepared, a human expert analyzes the samples with a microscope alone or with a microscope coupled to a camera and a computer, allowing both a more standardized and quantitative study. The microscope may be configured for fully automatic analysis, wherein the microscope is automated with a motorized stage and focus, motorized objective changers, automatic light intensity controls and the like.
The preparation of the samples for detection may involve different types of preparation techniques that are suited to microscopic imaging analysis, such as, for example, hybridization-based and immunolabeling-based preparation techniques. Such detection techniques may be coupled with appropriate revelation techniques, such as, for example, fluorescence-based and visible color reaction-based techniques.
In Situ Hybridization (“ISH”) and Fluorescent In Situ Hybridization (“FISH”) are detection and revelation techniques used, for example, for detection and quantification of genetic information amplification and mutation analyses. Both ISH and FISH can be applied to histological or cytological samples. These techniques use specific complementary probes for recognizing corresponding precise sequences. Depending on the technique used, the specific probe may include a chemical (ISH) marker or a fluorescent (FISH) marker, wherein the samples are then analyzed using a transmission microscope or a fluorescence microscope, respectively. The use of a chemical marker or a fluorescent marker depends on the goal of the user, each type of marker having corresponding advantages over the other in particular instances.
In case of protein expression analyses, further immunohistochemistry (“IHC”) and immunocytochemistry (“ICC”) techniques, for example, may be used. IHC is the application of immunochemistry to tissue sections, whereas ICC is the application of immunochemistry to cultured cells or tissue imprints after they have undergone specific cytological preparations, e.g. liquid based preparations. Immunochemistry is a family of techniques based on the use of specific antibody, wherein antibodies are used to specifically target molecules inside or on the surface of cells. The antibody typically, contains a marker that will undergo a biochemical reaction, and thereby experience a color change, upon encountering the targeted molecules. In some instances, signal amplification may be integrated into the particular protocol, wherein a secondary antibody that includes the marker stain follows the application of a primary specific monoclonal antibody.
In both hybridization and immunolabeling studies, chromogens of different colors are used to distinguish the different markers. As these markers may be cell compartment specific, this a priori knowledge can be used to automatically segment the cells (i.e. separates the nucleus masks from the cytoplasmic and or membrane masks). Overall, “colorimetric” algorithms are aimed to provide sample information to ease diagnosis and/or prognosis of the particular case. For illustration, the detection and quantification of the breast ER, PR and HER2 protein expression levels may be provided using a quantitative microscopy algorithm applied to immunohistochemistry (IHC) techniques.
In light of such image analysis techniques, however, there exists a need for improvements that facilitate flexibility in such analysis while providing a pathologist with accurate and useful information for allowing the pathologist to form an appropriate diagnosis and/or prognosis.